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(54) Remote copy system 

(57) In a system in which data employed by a com- 
puter 600, 700 is stored in a storage system 100, the 
storage system transfers this data to another storage 
system 1 90 and a copy of the data is maintained in the 
other storage system, consistency of the copy is main- 
tained even in the case of data written to the storage 
system by a computer, to which a write time is not ap- 
plied. 

A source storage system 100, when a write time is 
applied to a write request, records the write time and 
applies this write time to the received write data and, 
when no write time is applied, applies the recorded write 
time to the received write data and transfers the write 
data with this write time applied thereto, to another stor- 
age system 190. The target storage system 190 stores 
the write data in a logical volume 500 in the target stor- 
age system in accordance with the write time. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

[0001 ] The present invention relates to a storage sys- 
tem that stores data that is employed by a computer and 
that receives updating of data from a computer, and in 
particular relates to processing for maintaining copies 
of data between a plurality of storage systems. 

2. Description of the Related Art 

[0002] In Laid-open European Patent Application No. 
0672985, a technique is disclosed whereby the data that 
is employed by a computer is stored by a storage system 
and a copy of this data is stored in a separate storage 
system arranged at a remote location, while reflecting 
the write sequence of the data. In the processing indi- 
cated in Laid-open European Patent Application No. 
0672985, the source storage system that has received 
the write data from the primary host computer reports 
completion of reception of the write data to the primary 
host computer only after reception of the write data. Af- 
ter this, the primary host computer reads a copy of the 
write data from the source storage system. A write time, 
which is the time at which the write request in respect 
of the write data was issued, is applied to this write data 
and, when the write data is read by the primary host 
computer, the write time is also transferred to the prima- 
ry host computer. In addition, the primary host computer 
transfers the write data and the write time to the second- 
ary host computer. After receiving the write data and the 
write time, the secondary host computer writes informa- 
tion including the write time to a control volume in the 
storage system on the secondary side and, in addition, 
writes the write data in the target storage system in the 
write time sequence, with reference to the write times at 
which the various items of write data were presented. 
By writing the write data in the target storage system in 
the write time sequence, it is possible to maintain con- 
sistent data in the target storage system. 
[0003] If write data were to be reflected to the target 
storage system neglecting the write sequence (the op- 
eration of storing write data in the target storage system 
will hereinbelow be referred to as "reflecting 0 the data), 
for example in the case of a bank account database, in 
processing to transfer funds from an account A to an 
account B, it would not be possible to reproduce the deb- 
iting of the account A and the crediting of the account B 
as a single transaction and it would be possible for ex- 
ample for a period to occur in the target storage system 
in which the balance of the account B was credited be- 
fore debiting of the balance of the account A. If, in this 
case, some fault occurred in the source storage system 
rendering it unusable prior to debiting the balance of the 
account A in the target storage system, mismatching da- 



ta would be left in the target storage system, with the 
result that incorrect processing would be performed if 
business were to be subsequently continued using the 
secondary host computer. Consequently, by storing the 

s write data in the target storage system preserving the 
write sequence, consistent data can be maintained, 
making it possible to guarantee correctness of a se- 
quence of related operations in respect of related data. 
[0004] US Patent Number 6092066 discloses a tech- 

10 nique whereby the data that is used by a computer is 
stored in a storage system and, by copying the data that 
is stored in this storage system to a separate storage 
system arranged at a remote location, the data can be 
maintained in the separate storage system even if the 

15 first storage system has become unusable due to for ex- 
ample a natural disaster or fire. 

[0005] US Patent Number 6209002 discloses a tech- 
nique whereby data employed by a computer is stored 
in a storage system and, by copying the data that is 
20 stored in this storage system to a separate storage sys- 
tem arranged at a remote location, and additionally cop- 
ying the data that has been received by this separate 
storage system to a third storage system, a high level 
of redundancy can be obtained in respect of data. 

25 

SUMMARY OF THE INVENTION 

[0006] In the technique that is disclosed in Laid-open 
European Patent Application No. 0672985, consistency 

30 of the copy of data stored in the target storage system 
cannot be maintained unless the host computer applies 
a write time to the write data, since the write sequence 
is maintained using the write time applied to the write 
data by the host computer when the write data from the 

35 host computer is reflected to the target storage system. 
In the case of a so-called mainframe host computer, the 
write time is applied to the write request, but, in the case 
of a so-called open system host computer, the write time 
is not applied to the write request. Consequently, in the 

40 technique disclosed in Laid-open European Patent Ap- 
plication No. 0672985, consistency of the copy of the 
data stored in the target storage system with I/O from 
an open system host computer cannot be maintained. 
[0007] Also in the case of US Patent Number 6092066 

45 and US Patent Number 6209002, there is no disclosure 
concerning maintenance of consistency of a copy of da- 
ta stored in a target storage system when the host com- 
puters include an open system host computer. 
[0008] Accordingly, in a computer system in which da- 

50 ta that is employed by computer is stored in a storage 
system and the data that is stored in this storage system 
is transferred to a separate storage system so that a 
copy of the data is also held in this separate storage 
system, there is herein disclosed a technique for main- 

55 taining consistency of the copy of the data stored in the 
separate storage system (i.e. the target storage system) 
even in respect of data written to the storage system by 
a host computer that does not apply a write time to the 
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write data, such as an open system host computer. 
[0009] The system comprises a first storage device 
system having a first logical volume coupled to a com- 
puter and in which data received from the computer is 
stored and a second storage device system coupled to 5 
the first storage device system and having a second log- 
ical volume In which a copy of data stored in the first 
logical volume is stored. 

[001 0] The first storage device system applies time in- 
formation to the write data received from the computer 
and sends the write data and this time information to the 
second storage device system; the second storage de- 
vice system stores the write data received from the first 
storage device system in the second logical volume in 
accordance with the time information applied to this 
write data. 

[001 1 ] In a computer system in which data that is em- 
ployed by computer is stored in a storage system and 
the data that is stored In this storage system is trans- 
ferred to a separate storage system so that a copy of 
the data is also held in this separate storage system, it 
is thereby possible to maintain consistency of the copy 
of the data that is stored in the separate storage system 
(target storage system), even in the case of data stored 
in the storage system by a host computer that does not 
apply the write time to the write data, such as an open 
system host computer. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0012] 

Figure 1 is a view showing an example of the layout 
of a computer system according to embodiment 1 ; 
Figure 2 is a diagram showing an example of a log- 
ical volume group; 

Figure 3 is a flow diagram showing an example of 
processing in the case where a write request is re- 
ceived by a storage device A; 
Figure 4 is a view showing an example of group 
management information; 

Figure 5 is a view showing an example of write data 
management information for managing write data; 
Figure 6 is a ftow diagram showing an example of 
transfer processing of write data from the storage 
device A to a storage device B; 
Figure 7 is a view showing an example of remote 
logical volume information of a logical volume; 
Figure 8 is a view showing an example of arrived 
write time information; 

Figure 9 is a flow diagram showing an example of 
reflection processing of write data in the storage de- 
vice B; 

Figure 1 0 is a flow diagram showing another exam- 
ple of processing in the case where the storage de- 
vice A has received a write request; 
Figure 11 is a flow diagram showing another exam- 
ple of processing in the case where the storage de- 



vice A has received a write request; 
Figure 12 is a view showing an example of the lay- 
out of a computer system according to embodiment 

2; 

Figure 13 is a view showing an example of the lay- 
out of a computer system according to embodiment 

3; 

Figure 14 is a flow diagram showing another exam- 
ple of processing in the case where the storage de- 
vice A in embodiment 3 has received a write re- 
quest; 

Figure 15 is a flow diagram showing an example of 
processing in the case where the management soft- 
ware A gives instructions for deferring processing 
of a write request in respect of the storage device 
A and creation of a marker; 

Figure 16 is a view showing an example of marker 
number information; 

Figure 17 is a view showing another example of 
write data management information; 
Figure 18 is a flow diagram showing an example of 
transfer processing of write data from the storage 
device A in embodiment 3 to the storage device B; 
Figure 19 is a flow diagram showing an example of 
reflection processing of write data in the storage de- 
vice B in embodiment 3; 

Figure 20 is a flow diagram showing another exam- 
ple of reflection processing of write data in the stor- 
age device B in embodiment 3; and 
Figure 21 is a view showing an example of the lay- 
out of a computer system according to embodiment 
4. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0013] Embodiments of the present invention are de- 
scribed below. However, it should be noted that the 
present invention is not restricted to the embodiments 
described below. 

Embodiment 1 

[0014] Figure 1 is a view showing an example of the 
layout of a computer system according to a first embod- 
iment. 

[001 5] This system comprises a storage device (also 
referred to as a storage system) A1 00, a mainframe host 
computer A (also called MFA) 600, an open system host 
computer A700, a storage device B190, a mainframe 
host computer B (also referred to as MFB) 690 and an 
open system host computer B790. The storage devices 
A 100 and MFA 600 and the open system host A 700 
are respectively connected by I/O paths 900. The stor- 
age device B 190 and MFB 690 and open system host 
B 790 are also respectively connected by I/O paths 900. 
The MFB 690 and open system host B 790 are normally 
a standby system. The MFA 600, MFB 690 and open 
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system host A 700 and open system host B 790 are con- 
nected by a network 920. 

[001 6] The M FA 600 and M FB 690 include an OS 6 1 0 
and application software (APP) 620. Also, the open sys- 
tem host A 700 and open system host B 790 likewise 
include an OS 710 and APP 720. An I/O request issued 
from the APP of the MFA 600, MFB 690, open system 
host A 700, or open system host B 790 through the OS 
is issued to the storage device A 1 00 or storage device 
B 190 through the I/O path 900. In this case, software 
such as a DBMS is included in the APP 620 or APP 720. 
[001 7] The storage device A 1 00 comprises a control 
section 200, control memory 300 and cache 400. The 
control section 200 comprises a write data reception 
section A 21 0 and write data transfer section A 220. The 
control section 200 accesses the control memory 300 
and performs the following processing, utilizing the in- 
formation stored in the control memory 300. The cache 
400 comprises high-speed memory that chiefly stores 
the read data or write data so that the storage device A 
can achieve a high I/O processing performance by em- 
ploying the cache 400. It should be noted that, prefera- 
bly, these components are duplicated and provided with 
back-up power sources, for purposes of fault resistance 
and availability. 

[0018] The storage device B 190 also comprises a 
control section 200, control memory 300 and cache 400. 
The control section 200 comprises a write data recep- 
tion section B 211 and write data reflection instruction 
section 230 and write data reflection section 240. The 
role of the control memory 300 and cache 400 is the 
same as in the description of the storage device A 1 00 
above. 

[001 9] The storage device A 1 00 and storage device 
B 190 provide logical volumes 500 constituting data 
storage regions in respect of the MFA 600, open system 
host A 700, MFB 690 and open system host B 790. It is 
not necessary that a single logical volume 500 should 
constitute the single physical device; for example it 
could be constituted by a set of storage regions dis- 
persed on a plurality of magnetic disc devices. Also, a 
logical volume may have for example a mirror construc- 
tion or a construction that has redundancy such as for 
example a RAID construction, in which parity data is 
added. 

[0020] The storage device A 100 provides a logical 
volume 500 as described above; however, in the case 
of the MFA 600 and open system host A 700, the type 
of logical volume 500 that is provided is different from 
that provided in the case of the storage device A 100; 
also, the logical and/or physical interfaces of the I/O 
paths 900 are different. The same applies to the storage 
device B 190, MFB 690 and open system host B 790. 
The time of the write request 630 is included in the write 
request 630 from the MFA 600 as the write time 650, but 
is not included in the write request 730 from the open 
system host A 700. 

[0021 ] The storage device A 1 00 and the storage de- 



vice B 1 90 are connected by transfer paths 91 0. As will 
be described, the storage device A 1 00 and the storage 
device B 1 90 can hold a copy of the content of one log- 
ical volume in another logical volume. In this embodi- 
5 ment, a copy of the content of the logical volume 500 of 
the storage device A 100 is held in the logical volume 
500 of the storage device B 1 90; the content of the up- 
dating performed on the logical volume 500 of the stor- 
age device A 1 00 is also stored in the logical volume 
10 500 of the storage device B 190 by being sent to the 
storage device B 1 90 through the transfer path 91 0. As 
will be described, the storage device A 1 00 and the stor- 
age device B 200 hold management information regard- 
ing the copy, indicating the relationship between the log- 
's jcal volumes and maintenance of the copy referred to 
above is performed by using this management informa- 
tion. The relationship between the logical volumes and 
the relationship of the logical volume groups, to be de- 
scribed, is set by the user in accordance with the user's 
20 needs. 

[0022] In this embodiment, the relationships between 
the logical volumes are grouped. Figure 2 is a diagram 
showing an example of a group of logical volumes. The 
broken lines indicate the copy relationship between the 

25 logical volumes 500 or between the logical volume 
groups i.e. the correspondence relationship of the 
source and target. In this embodiment, the sequence of 
write data in the storage device A 1 00 and reflection in 
the storage device B 1 90 are managed in units of logical 

30 volume groups comprising a plurality of such logical vol- 
umes and allocation of the necessary resources for 
processing as described above is also performed in 
units of logical volume groups. 

[0023] If these are performed for each of the individual 

35 logical volumes, the large number of items to be man- 
aged makes the management process complicated and 
there is also a possibility of the resources required for 
this processing being increased, due to the large 
number of items to be processed. On the other hand, if 

^0 the entire storage device A 1 00 is treated as a unit, de- 
tailed management can no longer be performed. In par- 
ticular, since demands such as performance in regard 
to the logical volumes 500 differ greatly between a main- 
frame host and an open system host, it is desirable to 

45 arrange for example for manual control operations from 
the user in regard to processing and setting such as of 
tuning conditions to be accepted separately, by arrang- 
ing for such hosts to perform processing separately, di- 
vided into respective groups. By setting up logical vol- 

50 ume groups in this way, flexible copy processing man- 
agement can be provided in response to the require- 
ments of users or businesses. 

[0024] Next, processing of writing of data onto each 
logical volume 500, transfer of data to a storage device 
55 B 1 90 and processing for reflection of data in the storage 
device B 190 will be described for the case where the 
logical volumes 500 that are used by the MFA 600 and 
the open system host A 700 are arranged to belong to 
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different logical volume groups. By means of these proc- 
esses, reflection to a copy is performed in write se- 
quence between the various logical volumes of the stor- 
age device A 100 and, regarding consistency between 
copies, it is arranged that mutual consistency can al- 
ways be maintained between the mainframe host data 
and open system host data. 

[0025] Figure 3 is a view showing the processing that 
is performed in the case where a write request is re- 
ceived from the MFA 600 or open system host A 700 in 
respect of a logical volume 500 (logical volume 500 con- 
stituting the source) where a copy of the logical volume 
500 is being created. The write data reception section 
A 210 receives a write request from the MFA 600 or open 
system host A 700 (step 1000). If the write time 650 is 
included in the write request that is received (step 1 001 ), 
the write data reception section A 210 stores the write 
data in the cache 400 (step 1002) and creates (step 
1 003) write data management information 330 by apply- 
ing (assigning) a sequential number to the write data. 
The write data reception section A 21 0 then records the 
write time 650 in the write data management information 
330. Also, when the sequential number is applied, the 
write data reception section A 210 obtains the sequen- 
tial number from the group management information 
31 0 of the logical volume group to which the logical vol- 
ume that is being written belongs and records a value 
obtained by adding 1 thereto in the write data manage- 
ment information 330 as the sequential number of the 
write data, and records this new sequential number in 
the group management information 310. 
[0026] Figure 4 is a view showing an example of group 
management information 31 0 of the various logical vol- 
ume groups. The group ID is the ID for identifying a log- 
ical volume group in the storage device A 1 00. The se- 
quential numbers are numbers that are continuously 
given to write data in respect of a logical volume belong- 
ing to the logical volume group in question. Numbers 
successively increased by 1 in each case are applied to 
such write data, the initial value being for example 0. 
The logical volume number is the number of the logical 
volume that belongs to the logical volume group in ques- 
tion. The logical volume number is the ID of the logical 
volume belonging to the logical volume group in ques- 
tion in the storage device A 100. The remote storage 
device ID has a logical volume group that is paired with 
the logical volume group in question and is an ID (e.g. 
serial number) that specifies the storage device (in this 
embodiment, the storage device B 190) where a copy 
of the content of the logical volume belonging to the log- 
ical volume group in question is stored. The remote 
group ID is an ID that specifies the logical volume group 
that is paired with the logical volume group in question 
in the remote storage device (storage device B 190) i. 
e. the logical volume group to which the logical volume 
500 (also called the remote logical volume) belongs in 
which a copy of the content of the logical volume be- 
longing to the logical volume group in question is stored. 



[0027] Figure 5 is a view showing an example of write 
data management information 330 for managing the 
various write data. The logical volume ID is the ID of the 
logical volume in which the write data is stored. The write 

5 address is the write start address of the write data in 
question in the aforesaid logical volume. The write data 
length is the length of the write data in question. The 
write data pointer is the storage start address of the write 
data in question in the cache 400. The sequential num- 

10 bers are numbers that are continuously given to write 
data in the logical volume group and to which the logical 
volume belongs in which the write data is written. The 
write time will be discussed below. The "transfer re- 
quired" bit is a bit that indicates whether or not the write 

15 data in question needs to be transferred to the storage 
device B and is set to ON when write data management 
information 330 is created by receipt of write data by the 
write data reception section A 21 0. The write data man- 
agement information 330 is managed in the form of a 

20 list for example for each logical volume group. 

[0028] Returning to Figure 3, in step 1004, the write 
data reception section A 210 records the write time 650 
as the write time information 340 in the control memory 
300. 

25 [0029] If, in step 1 001 , no write time is included in the 
write request, the write data reception section A 210 
stores the write data in the cache 400 (step 1005) and 
obtains from the write time information 340 a write time, 
which it applies (assigns) to the write data, and creates 

30 write data management information 330 (step 1 006) by 
. applying a sequential number obtained from the group 
management information 31 0. At this time, the write da- 
ta reception section A 21 0 then records the time at which 
the write time information 340 was recorded, as the write 

35 time of the write data management information 300, and 
finds a sequential number by the same procedure as in 
the case of step 1 003 described above and records this 
sequential number in the write data management infor- 
mation 300. 

40 [0030] Finally, in step 1007, completion of writing is 
reported to the MFA 600 or to the open system host A 
700. The aforesaid processing does not include the 
time-consuming processing of physically writing the 
write data that is stored in the cache 400 to the recording 

45 medium of the logical volume 500 or of transferring the 
write data to the storage device B 190; this processing 
is performed subsequently in asynchronous fashion, 
with an appropriate timing. Consequently, the time re- 
quired until reporting of completion of writing after re- 

50 ceiving the write request by the write data reception sec- 
tion A 21 0 need only be a short time, so rapid response 
to the MFA 600 or open system host A 700 can be 
achieved. 

[0031] Figure 6 is a view showing an example of trans- 
55 fer processing of write data to the storage device B 1 90 
from the storage device A 100. The write data transfer 
section A 220 finds (step 1100) the information relating 
to the write data that is transferred to the storage device 
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B 190 by referring to the list of the write data manage- 
ment information 330 to find the write data that needs 
to be transferred and, in addition, referring to the write 
data management information 330, group management 
information 310 and remote logical volume information 
320. This information includes the write address ac- 
quired from the write data management information 330, 
the write data length, the sequential number, the write 
time, the remote storage device ID acquired from the 
remote logical volume information 320, the remote log- 
ical volume number, and the remote group number ob- 
tained from the group management information 31 0 us- 
ing the logical volume ID. 

[0032] Figure 7 is a view showing an example of the 
remote logical volume information 320 of the various 
logical volumes. The logical volume ID is the ID of the 
logical volume on the source side (logical volume 500 
included in the storage device A 1 00 in embodiment 1 ). 
The remote storage device ID is an ID (for example a 
serial number) specifying the storage device (storage 
device B 190 in embodiment 1) having the logical vol- 
ume (also called the remote logical volume) in which is 
stored a copy of the data stored by the logical volume 
in question that is paired with the logical volume in ques- 
tion. The remote logical volume ID is an ID that specifies 
the remote logical volume (i.e. the logical volume 500 
on the target side, where a copy of the data that was 
stored in the logical volume is stored) in the remote stor- 
age device (storage device B 190 in embodiment 1). 
[0033] Next, returning to Figure 6, the write data trans- 
fer section A 220 transfers (step 1101) to the storage 
device B 190 the write data and the information found 
in step 1100. The write data reception section B 211 of 
the storage device B stores (step 1102) the received 
write data and information in the cache 400 and creates 
(step 1103) write data management information 330 
from the received information. The items of the write da- 
ta management information 330 of the storage device 
B 1 90 are the same as the items of the write data man- 
agement information 330 of the storage device A 100. 
The content of the write data management information 
330 of the storage device B 1 90 differs from that of the 
write data management information 330 of the storage 
device A 100 in that the logical volume ID is the ID of 
the logical volume 500 on the target side where the copy 
is stored and the write data pointer is the storage start 
address of the write data in the cache 400 of the storage 
device B 190 and the "transfer needed" bit is normally 
OFF, but is otherwise the same. 

[0034] The storage device B 1 90 also has group man- 
agement information 31 0, but the items thereof are the 
same as in the case of the storage device A 100. Re- 
garding the content of the group management informa- 
tion 310, the group ID is an ID that specifies the logical 
volume group to which the logical volume 500 on the 
side of the target where the copy is stored belongs, the 
remote storage device ID is the ID of the storage device 
(storage device A 1 00 in the case of embodiment 1 ) con- 



stituting the source and the remote group ID is an ID 
that specifies the logical volume group to which the re- 
mote logical volume (i.e. the logical volume 500 consti- 
tuting the source) belongs in the remote storage device 
5 (storage device A 100 in embodiment 1). The storage 
device B 1 90 also has remote logical volume information 
320, but the items thereof are the same as in the case 
of the storage device A 100 and, regarding its content, 
the logical volume ID is an ID that specifies the logical 
10 volume 500 where the copy is stored, the remote stor- 
age device ID is an ID that specifies the ID of the storage 
device (storage device A 100) constituting the source 
and the remote logical volume ID is an ID that specifies 
the remote logical volume (logical volume 500 constitut- 
es jng the source) in the remote storage device (storage 
device A 100). 

[0035] Returning to Figure 6, next, the write data re- 
ception section B 21 1 updates the arrived write time in- 
formation 350 (step 1104). 

20 [0036] Figure 8 is a view showing an example of ar- 
rived write time information 350 of the various groups. 
The group ID is an ID that specifies the logical volume 
group in the storage device B 1 90. The latest write time 
of the arrived write data is the latest time closest to the 

25 current time, of the write times applied to the write data 
received by the write data reception section of B 211 , in 
respect of the logical volume groups of the storage de- 
vice B 190. However, if it appears, from the sequential 
number order, that some of the write data has not yet 

30 arrived (some of the sequence of write data is missing), 
the latest time of the write time applied to these items of 
write data is recorded as the arrived write data time in- 
formation, taking the continuous time comparison range 
in the order of the sequential numbers as being up to 

35 the final write data (write data immediately preceding the 
missing data). 

[0037] In transfer of the write data between the write 
data transfer section A 220 and the write data reception 
section B 211 , a plurality of items of write data may be 

40 simultaneously transferred in parallel. The write data is 
therefore not necessarily received in the write data re- 
ception section B 21 1 in the order of the sequential num- 
bers but, as will be described, the write data is reflected 
in the order of the sequential numbers to each of the 

45 logical volume groups (i.e. it is stored in the logical vol- 
umes of the storage device B 1 90), so the write data is 
reflected to the copy in the order of updating (i.e. in the 
order of writing of the write data in the storage device A 
100). 

50 [0038] Returning once more to Figure 6, finally, the 
write data reception section B 211 reports completion of 
reception of the write data to the write data transfer sec- 
tion A 220 (step 1105). The write data transfer section 
A 220 of the storage device A 1 00 that has received this 

55 write data turns the "transfer required" bit of the write 
data management information 330 OFF in respect of the 
write data corresponding to the report of completion of 
reception of write data. At this time, the storage device 
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A 1 00 may discard from the cache the arrived write data 
that was held for transfer to the storage device B 190. 
[0039] Figure 9 it is a view showing an example of the 
reflection processing of write data in the storage device 
B 1 90 (i.e. the processing of storage of the write data to 5 
the logical volume). 

[0040] The write data reflection instruction section B 
230 checks the arrived write time information 350 of all 
the logical volume groups of the storage device B 1 90 
and finds, of these, the earliest time (step 1200). The 
write data reflection instruction section B 230 gives in- 
structions (or permission) (step 1201) to the write data 
reflection section B 240 for reflection to these logical vol- 
umes of the write data whose write time is previous to 
the time that was thus found. When the write data re- 
flection section 240 receives these instructions (or per- 
mission), by referring to the write data management in- 
formation 330 and group management information 31 0, 
it reflects the write data in the designated time range (i. 
e. the write data whose write time is previous to the time 
found in step 1 200), in the order of the write times, or, if 
these write times are the same, in the order of the se- 
quential numbers in the various logical volume groups, 
in respect of the logical volume 500 in which the copy is 
stored (i.e. the write data is stored in the logical volume 
on the target side) (step 1202). After completion of re- 
flection of all of the write data in the range specified in 
step 1202, the write data reflection section B 240 reports 
completion of the instructed processing (step 1203) to 
the write data reflection instruction section 230. The 
storage device B may discard the reflected write data 
from the cache 400. 

[0041] By means of the above processing from step 
1200 to step 1203, one of cycle of reflection processing 
is completed. The write data reflection instruction sec- 
tion B 230 and the write data reflection section B 240 
repeat the above cycle in order to reflect the write data 
transferred from the storage device A continuously. 
[0042] By means of the above processing, a copy of 
the updated data of the storage device B 1 90 is stored 
maintaining the order between updating of data by the 
mainframe host and updating of data by the open sys- 
tem host. Regarding data consistency between the cop- 
ies, mutual consistency can be maintained between the 
data of the mainframe host and the data of the open sys- 
tem host. 

[0043] Specifically, the storage device A 100 utilizes 
the write time 650 contained in the write request 630 
received from the mainframe host and applies a write 
time also to the write data received from the open sys- 
tem host and, furthermore, manages the received write 
data using both the write times and the sequential num- 
bers. The target storage device B 190 designates the 
write data that is capable of being reflected (i.e. that is 
capable of storage in a logical volume on the target side) 
using the sequential numbers and the write times and 
stores the designated write data in a logical volume on 
the target side. As a result, even if buffering and/or trans- 



ferring are provided in parallel mid-way, write order is 
maintained between the data written from the main- 
frame host and the data written from the open system 
host, so copy data can be stored in a logical volume of 
the storage device B 190 on the target side. 
[0044] Also, even if some fault occurs in for example 
the storage device A 100, so that previously updated 
write data does not reach the storage device B 190, 
since the sequential numbers will not be continuous in 
respect of the write data of write times subsequent to 
the write time of the write data that failed to arrive, re- 
flection thereof will not be allowed. Gaps of updating of 
data cannot therefore occur in the target side storage 
device B 190 and consistency between the source stor- 
age device A 100 and target storage device B 190 is 
ensured. As a result, even if a fault occurs in the source 
storage device A 1 00, business can be continued using 
the content of the logical volume 500 of the storage de- 
vice B 190, which is matched with the MFB 690 and/or 
open system host B 790. 

[0045] Also, since, in the above processing, write 
times are applied to all of the write data received by the 
storage device A 100, irrespective of whether the host 
that employs the data is a mainframe host or open sys- 
tem host, it is possible to ascertain information such as 
up to which write time the write data in any desired log- 
ical volume 500 has been transferred from the storage 
device A 1 00 to the storage device B 1 90 or has arrived 
at the storage device B 1 90 or has been reflected at the 
storage device B 190 (i.e. has been stored in a logical 
volume). 

[0046] It should be noted that, in order to lighten the 
processing load in the above step 1202, the write data 
in the designated time range may be stored in the logical 
volume 500 that stores the copy in sequential number 
order in the various logical volume groups, neglecting 
the write time order. In this case, consistency between 
the copies (i.e. between the logical volumes of the stor- 
age device B 190 on the target side) is maintained by 
the timing of the reports of completion of processing in 
step 1203. If it is desired to hold consistent data of the 
period between a report of completion of processing and 
the next report of completion of processing, a snapshot 
of the logical volume 500 in which the copy is stored 
may be acquired with the timing of the report of comple- 
tion of processing. The technique disclosed in for exam- 
ple US Patent 6658434 may be employed as a method 
of acquiring such a snapshot. In this method, the storage 
content of a logical volume 500 (source volume) in which 
is stored the data whereof a snapshot is to be acquired 
is copied to another logical volume 500 (target volume) 
of the storage device B 1 90, so that the updated content 
is reflected also to the target volume when the source 
of volume is updated. However, in this embodiment, 
once the snapshot of the source volume has been 
stored in the target volume, the content of the target vol- 
ume is frozen and verified by stopping reflection at that 
time. 
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[0047] Also in the transfer processing of the above 
write data, it was assumed that, initially, the write data 
transfer section A 220 transfers the write data in respect 
of the write data reception section B 211; however, it 
would be possible for the write data reception section B 
211 to initially issue a write data transfer request in re- 
spect of the write data transfer section 220 and for the 
write data transfer section A 220 to transfer the write 
data in respect of the write data reception section B 21 1 
after having received this request. By employing write 
data transfer requests, the pace of transfer of write data 
can be adjusted in accordance with for example the 
processing condition or toad of the storage device B 1 90 
or the amount of write data that has been accumulated. 
[0048] Also, in the above processing, it was assumed 
that the location of storage of the write data was the 
cache 400; however, by preparing a separate logical vol- 
ume 500 for write data storage, the write data could be 
stored in this logical volume 500. In general, a logical 
volume 500 of large volume may be prepared in respect 
of the cache 400, so this makes it possible for more write 
data to be accumulated. 

[0049] Also, in the above processing, it was assumed 
that the write time information 340 was updated by the 
write time 650 of reception from the mainframe host; 
however, it may be arranged for the storage device A 
100 to possess an internal clock and to constantly up- 
date the write time information 340 by reference to this 
clock. In this case, Figure 10 shows an example of the 
processing that is executed when a write request in re- 
spect of a logical volume 500 (logical volume 500 con- 
stituting the source) where the storage device A 1 00 cre- 
ates a copy is received from the MFA 600 or open sys- 
tem host A 700. This processing is processing corre- 
sponding to the processing shown in Figure 3. 
[0050] The write data reception section A 210 re- 
ceives (step 1300) a write request from the MFA 600 or 
open system host A 700. The write data reception sec- 
tion A 21 0 stores (step 1 301 ) the write data in the cache 
400 and applies a write time to the write data by referring 
to the write time information 340 that is constantly up- 
dated in accordance with the clock provided in the stor- 
age device A 100, and creates (step 1302) write data 
management information 330 by applying a sequential 
number to the write data, by referring to the group man- 
agement information 310. Finally, completion of writing 
is reported to the MFA 600 or open system host A 700 
(step 1303). 

[0051] Also, in the above processing, a time is used 
in the write time information 340 or the write time of the 
write data management information 300 or the arrived 
write time information 350; however, the time that is em- 
ployed for this purpose need not necessarily be of the 
form of years, months, days, hours, minutes, seconds, 
milliseconds, microseconds, nanoseconds or a total of 
an ordinary time and instead a sequential number could 
be employed. In particular, Figure 11 shows an example 
of the processing when the storage device A 100 has 



received a write request in respect of the logical volume 
500 (logical volume 500 constituting the source), where 
the copy is created, from the MFA 600 or open system 
host A 700, in a case where the storage device A 100 

5 itself updates the write time information 340. This 
processing is processing corresponding to Figure 3 or 
Figure 1 0. It should be noted that, in Figure 1 1 , the initial 
value of the write time information 340 may for example 
be 0 and numbers successively incremented by 1 may 

10 be applied to the write data as shown below as the write 
times. 

[0052] The write data reception section A 210 re- 
ceives a write request (step 1400) from the MFA 600 or 
open system host A 700. The write data reception sec- 

15 tion A 210 stores the write data in the cache 400 (step 
1 401 ), reads the number from the write time information 
340 and applies to the write data (step 1 402) as the write 
time the value obtained by incrementing this by 1 . Then 
the write data reception section A 21 0 records the value 

20 after incrementing by 1 as the write time information 
340, thereby updating the write time information 340 
(step 1 403). The write data reception section A 21 0 also 
creates the write data management information 330 
(step 1 405) by applying a sequential number to the write 

25 data (step 1404) by referring to the group management 
information 310. The write data reception section A 21 0 
finally reports completion of writing (step 1406) to the 
MFA 600 or open system host A 700. 
[0053] When a sequential number is employed as the 

30 write time in this manner, in the storage device B 190, 
instead of the write data reception section B 211 being 
arranged to update the arrived write time information 
350 using the write time applied to the write data re- 
ceived and the write data reflection instruction section 

35 B 230 being arranged to designate the range of write 
data capable being stored in a logical volume of the stor- 
age device B by checking the arrived write time infor- 
mation 350 of the various logical volume groups, it may 
be arranged for the write data reflection section 240 to 

40 reflect (i.e. store) the write data arriving at the storage 
device B by referring to the sequential number recorded 
at the write time of the write data management informa- 
tion 330 in the logical volume 500 without skipping num- 
bers in the number sequence. 

45 

[Embodiment 2] 

[0054] Figure 1 2 is a view showing an example of the 
layout of a computer system according to a second em- 

50 bodiment. 

[0055] The differences with respect to embodiment 1 
lie in that the MFA 600 and open system host A 700 are 
connected with the storage device C 180 through an I/ 
O path 900 and the storage device C 1 80 is connected 

55 with the storage device A 1 00 through a transfer path 
910. In this embodiment, a copy of the data stored in the 
logical volume 500 of the storage device C 1 80 is stored 
in a logical volume 500 of the storage device A 1 00. Fur- 
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ther, a copy of the data stored in the logical volume 500 
of the storage device A is stored in the logical volume 
500 of the storage device B 190 in processing like the 
processing described in embodiment 1 . That is, in this 
embodiment, a copy of the data stored in the logical vol- 
ume 500 of the storage device C 180 is stored in the 
storage device A 100 and the storage device B 190. In 
order to implement such processing, the storage device 
C 180 is provided with the various items of information 
and a construction like that of the storage device A 1 00 
described in embodiment 1 . 

[0056] When the storage device C 1 80 has received 
a write request 630 or a write request 730 for the logical 
volume 500 from the MFA 600 or open system host A 
700, it stores the received write data 640 or write data 
740 in a logical volume in the storage device C 180 and 
transfers this to the write data reception section A 210 
of the storage device A 100. At this point, in contrast to 
the processing described in embodiment 1 , the storage 
device C 1 80 sends notification of completion of writing 
to the MFA 600 or open system host A 700 after waiting 
for notification of completion of reception from the write 
data reception section A 210, and the storage device C 
1 80 is thereby able to guarantee that a copy of the write 
data 640 or write data 740 that was written thereto is 
present in the storage device A 100. In this way, if for 
example due to the occurrence of some fault in the stor- 
age device C 1 80 oron the transmission path 91 0, trans- 
fer of data to the storage device A 1 00 has not succeed- 
ed, the MFA 600 or open system host A 700 will not 
deem write data that have not been transferred to the 
storage device A 1 00 to have been written but will only 
deem write data that have been received by the storage 
device A 100 to have actually been written; a copy as 
expected by the APP 620 on the MFA 600 or the APP 
720 on the open system host A 700 will therefore exist 
on the storage device A 100. Furthermore, after all of 
the write data received by the storage device A 1 00 have 
been sent to the storage device B 190, a copy as ex- 
pected will also exist on the storage device B 190, so, 
at the time where the processing executed by the MFA 
600 or open system host A 700 was interrupted, the 
M FB 690 or open system host B 790 will be able to con- 
tinue business using data as expected identical with the 
data that are recognized as having been written by the 
MFA 600 or open system host A 700. 
[0057] As initially indicated in embodiment 1, when 
the write time information 340 is updated by the write 
time 650 applied to the write data, the write data recep- 
tion section C 21 2 of the storage device C 1 00, if a write 
time 650 is included in the received write request 630, 
records the write time also in the write data management 
information 330 and the write data transfer section C 
222 also transfers this write time to the write data recep- 
tion section A21 0 of the storage device A 1 00 when per- 
forming write data transfer. After receiving the write data 
and the write time, the write data reception section A 
2 1 0 processes the write data and the write time received 



from the storage device C 180 by the same method as 
the processing of the write request 630 that was re- 
ceived from the mainframe host in embodiment 1 ; con- 
sistency between the copies stored in the logical vol- 

5 umes in the storage device A 1 00 is thereby maintained 
and consistency between the write data issued from the 
mainframe host and the write data issued from the open 
system host can thereby be maintained. 
[0058] In this way, even if, due for example to a large- 

10 scale disaster, faults occur in both of the storage device 
C 180 and the storage device A 100, business can be 
continued using the consistent content of the logical vol- 
ume 500 of the storage device B 190, which was 
matched with the MFB 690 and open system host B 790. 

15 As indicated in the final part of embodiment 1 , when the 
write time information 340 is updated from the storage 
device A 100 itself, transfer of the write time from the 
storage device C 180 is unnecessary, so that, after re- 
ceiving the write data from the storage device C 180, 

20 the write data reception section A 210 may perform 
processing on the write data like the processing of Fig- 
ure 11 indicated in the latter part of embodiment 1 . 
[0059] It should be noted that there may be a plurality 
of storage devices C 180 that connect to the storage 

25 device A 100. 

[0060] Also, although not shown, if the mainframe 
host and open system host are connected by an I/O path 
with the storage device A 100, the. mainframe host or 
open system host that is connected with the storage de- 

30 vice A may continue the business that was being con- 
ducted by the MFA 600 or open system host A 700 using 
the consistent content of a logical volume 500 of the 
storage device A 1 00 that was matched therewith, in the 
event that a fault occurs in the M FA 600 or open system 

35 host A 700 or storage device C 1 80. 

[Embodiment 3] 

[0061 ] Figure 1 3 is a view showing an example of the 
40 construction of a computer system according to Embod- 
iment 3. 

[0062] The chief differences with respect to embodi- 
ment 1 lie in that there are a plurality of respective stor- 
age devices A 100 and storage devices B 190, the MFA 

45 600 and open system host A 700 are connected through 
an I/O path 900 respectively with a plurality of storage 
devices A 100, the MFB 690 and the open system host 
B 790 are connected through an I/O path 900 respec- 
tively with a plurality of storage devices B 1 90, the MFA 

so 600 includes management software A 800 and the M FB 
690 includes management software B 890. Other differ- 
ences will be described below. 

[0063] Hereinbelow, the processing in respect of writ- 
ing performed to the various logical volumes 500, trans- 
55 fer of write data to the storage device B 1 90 and the 
processing of reflection of write data in the storage de- 
vice B 190 (i.e. storage of the write data in the logical 
volume) will be described in respect of the logical vol- 
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umes 500 employed by the MFA 600 and the open sys- 
tem host A 700. This processing ensures that mutual 
consistency is maintained between the data of the main- 
frame host and the data of the open system host in re- 
gard to consistency between copies respectively stored 
in the plurality of logical volumes that are possessed by 
the plurality of storage devices B 190. 
[0064] Figure 1 4 is a view showing an example of the 
processing when a write request in respect of the logical 
volume 500 (logical volume 500 constituting the source) 
in which a copy is created by the storage device A 1 00 
is received from the MFA 600 or open system host A 
700. 

[0065] The write data reception section A 210 re- 
ceives (step 1500) a write request from the MFA 600 or 
open system host A 700. The write data reception sec- 
tion A 210 stores the write data in the cache 400 (step 
1501) or, as in embodiment 1, creates write data man- 
agement information 330 (step 1502) by acquiring a se- 
quential number by referring to the group management 
information 31 0. Finally, the write data reception section 
A 210 reports to the MFA 600 or open system host A 
700 completion of writing (step 1503). The group man- 
agement information 31 0 is the same as that in the case 
of embodiment 1 . The write data management informa- 
tion 330 of this embodiment will be described later. 
[0066] Figure 1 5 is a view showing an example of the 
processing when the management software A 800 gives 
instructions for deferment of processing of write re- 
quests in respect of the storage device A 1 00 and cre- 
ation of a marker. As will be described later, consistency 
is established between the copies stored in the plurality 
of storage devices B 190 by subsequently performing 
synchronization of reflection to the copies, with the tim- 
ing with which this processing was performed during up- 
dating of the logical volume 500 of the storage device A 
100. 

[0067] First of all, the management software A 800 
gives instructions for deferment of processing of write 
requests to all of the storage devices A 1 00 (step 1 600). 
On receipt of these instructions, the write data reception 
section A 210 defers processing of write requests (step 

1601) and reports to the management software A 800 
the fact that deferment has been commenced (step 

1602) . After the management software A 800 has con- 
firmed that commencement of deferment has been re- 
ported from all of the storage devices A 100 that have 
been so instructed, processing advances to the follow- 
ing processing (step 1603 and step 1604). 

[0068] Next, the management software 800 instructs 
all of the storage devices A 1 00 to create markers (step 
1605). This instruction includes a marker number as a 
parameter. The marker number will be described sub- 
sequently. On receipt of this instruction, the marker cre- 
ation section A 250 records the received marker number 
in the marker number information 360 shown in Figure 
16 stored in the control memory 300 (step 1606) and 
creates (step 1607) special write data (hereinbelow 



called a marker) for information transmission in respect 
of all of the logical volume groups. A marker is write data 
in which a marker attribute is set in the write data man- 
agement information 300. 
5 [0069] Figure 17 is a view showing an example of 
write data management information 330 of write data in 
this embodiment; a marker attribute bit and marker 
number are added to the write data management infor- 
mation 330 of embodiment 1 . 
w [0070] The marker attribute bit is a bit indicating that 
the write data in question is a marker and is OFF in the 
case of ordinary write data but is set to ON in the case 
of a marker. A marker number as described above is set 
in the "marker number". The sequential number in the 
is group is acquired and applied in respect of a marker in 
the same way as in the case of ordinary write data. Spe- 
cifically, in marker creation, the marker creation section 
A 250 obtains a sequential number from the group man- 
agement information 310 of the group in the same way 

20 as in the processing of the write data reception section 
A 21 0 and records a value obtained by adding 1 thereto 
in the write data management information 330 as the 
sequential number of the aforesaid marker, and records 
the new sequential number in the group management 

25 information 310. When the sequential number has been 
applied in this way to the marker, it is transferred to the 
storage device B 1 90 in the same way as in the case of 
ordinary write data, but the marker is not reflected to the 
logical volume 500. 

30 [0071] The marker number is a number for identifying 
the instruction in response to which the marker was cre- 
ated; when a marker creation instruction is issued by the 
management software A 800, for example the initial val- 
ue thereof is 0 and the marker number is incremented 

35 by 1 before being issued. The management software A 
800 may confirm the current marker number by reading 
the marker number recorded in the marker number in- 
formation 360. 

[0072] Returning to Figure 1 5, after the marker crea- 
te tion section A 250 has created a marker in respect of ail 
of the logical volume groups, the marker creation sec- 
tion A 250 reports completion of marker creation to the 
management software A 800 (step 1608). After confirm- 
ing that completion of marker creation has been report- 
4 5 ed from all of the designated storage devices A 1 00, the 
management software A 800 proceeds to the subse- 
quent processing (step 1609, step 1610). 
[0073] The management software A 800 gives in- 
structions (step 1611) for cancellation of deferment of 
50 processing of write requests to all of the storage devices 
A 100. On receipt of these instructions, the write data 
reception section A 21 0 cancels deferment of process- 
ing of write requests (step 1612) and reports to the man- 
agement software A 800 (step 1613) the fact that such 
55 deferment has been cancelled. 

[0074] Figure 18 is a view showing an example of 
transfer processing of write data to a storage device B 
1 90 from a storage device A 1 00. This processing is sub- 
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stantially the same as the transfer processing described 
in Figure 6 of embodiment 1 , but differs in that no up- 
dating of the arrived write time information 350 is per- 
formed by the write data reception section B 211. It 
should be noted that the write data management infor- 
mation 330 of the storage device B 190 is the same as 
the write data management information shown in Figure 
1 7, described above; in step 1 703, the presence or ab- 
sence of the marker attribute of the write data and/or the 
marker number recorded in the write data management 
information 330. 

[0075] Figure 1 9 is a view showing an example of the 
processing of reflection (storage) of write data to a log- 
ical volume in the storage device B 190. First of all, the 
management software B 890 gives instructions for re- 
flection of the write data, as far as the marker, to the 
logical volume 500 in which a copy is stored (step 1 800) 
to all of the storage devices B 1 90. After receiving such 
an instruction, the write data reflection section B 240 re- 
fers to the write data information 330 and group man- 
agement information 310 and reflects (step 1801) the 
write data as far as the marker, in the sequential number 
order in each group, to the logical volume 500 in which 
the copy is stored. Specifically, the write data reflection 
section B 240 continues to store the write data in the 
logical volume in the order of the sequential numbers, 
but stops data storage processing on finding write data 
with the marker attribute (i.e. a marker) and then reports 
completion of reflection to the management software B 
890 (step 1802). In the aforementioned processing, the 
write data reflection section B 240 checks the marker 
numbers of the markers that are recorded in the write 
data management information 330 and thereby ascer- 
tains whether the marker number is correct (whether the 
marker conforms to rules which are the same as the 
marker number decision rules, described above, for ex- 
ample of being a number whose initial value is 0 and 
that is incremented by 1 with respect to the previous 
marker number). If the marker number is not correct, the 
write data reflection section B 240 reports an abnormal 
situation to the management software B 890; if the mark- 
er number is correct, the write data reflection section B 
240 records the marker number in the marker number 
information 360 and reports a normal situation. The 
management software B 890 may confirm the current 
marker number by reading the marker number that is 
recorded in the marker number information 360. 
[0076J After confirming that a "normal reflection com- 
pleted" report has been obtained from all of the storage 
devices B 190 that had been designated, the manage- 
ment software B 890 proceeds to the next processing 
(step 1803, step 1804). 

[0077] Next, the management software B 890 gives 
instructions (step 1805) for updating of the snapshot of 
the logical volume 500 that stores the copy to all of the 
storage devices B 190. After receiving this instruction, 
the snapshot acquisition section B 260 updates (step 
1806) the snapshot of the content of the logical volume 



500. As the method of acquiring such a snapshot, for 
example the technique disclosed in US Patent Number 
6658434 may be employed. It should be noted that, in 
this embodiment, just as in the case of the method de- 

5 scribed in embodiment 1 , reflection of the write data to 
the volume that stores the snapshot data is stopped at 
the time of acquisition of the snapshot, and the content 
of the volume that stores the snapshot is frozen. After 
updating the snapshot, the snapshot acquisition section 

10 B 260 reports completion of snapshot updating to the 
management software B 890 (step 1 807). After confirm- 
ing that a report of completion of snapshot updating has 
been obtained from all of the storage devices B 1 90 that 
were designated, the management software B 890 pro- 

15 ceeds to the next processing (step 1808, step 1809). 
[0078] The management software A 800 and the 
management software B 890 respectively repeat the 
processing of the aforesaid step 1 600 to step 1613 and 
of step 1 800 to step 1 809. In this way, the updating of 

20 the storage device A 100 to the logical volume 500 is 
constantly reflected to the logical volume 500 of the stor- 
age device B 190. 

[0079] By processing as described above, the data 
updating by the MFA 600 and the open system host A 

25 700 is stopped and a marker is created with the timing 
(checkpoint) at which the updating condition is unified 
between the plurality of storage devices; reflection (i.e. 
storage) of the updated data to the stored copy data in 
the plurality of target logical volumes provided in the plu- 

30 rality of target storage devices B 190 can be synchro- 
nized at the time immediately preceding the writing of 
the marker, so mutual consistency between the various 
copies can be obtained with the data of the mainframe 
host and the data of the open system host at the time of 

35 this marker. In addition, the MFB 690 or open system 
host B 790 can continue business using the matched 
data stored in the snapshot volume, since a copy having 
mutual consistency is held in the snapshot volume, this 
snapshot being acquired by reflection of the updated da- 

40 ta to the copy data at a time that is synchronized be- 
tween the plurality of copy data. 

[0080] In the above processing, the sn apshot was as- 
sumed to be updated by the storage device B 1 90 in 
response to an instruction from the management soft- 
45 ware B 890, but it would be possible to update the snap- 
shot with the timing of synchronization of reflection of 
the updated data between the copy data of a plurality of 
storage devices B 190. Figure 20 shows an example of 
the reflection processing of write data to the copy in the 
50 storage devices B 190 in this case. 

[0081] The management software B 890 gives in- 
structions (step 1 900) for reflection of the write data as 
far as the marker to the logical volume of 500 that stores 
the copy in all of the storage devices B 1 90. After receiv- 
es jng such an instruction, the write data reflection section 
B 240 reflects the write data in the same way as in the 
processing described with reference to Figure 19 but 
stops the reflection as soon as it finds a marker and no- 
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tifies the snapshot acquisition section B 260 (step 1 901 ). 
After receiving such notification, the snapshot acquisi- 
tion section B 260 updates the snapshot of the content 
of the logical volume 500 and notifies the write data re- 
flection section B 240 (step 1902). After receiving this s 
notification, the write data reflection section B 240 re- 
ports completion of reflection to the management soft- 
ware B 890 (step 1903). The management software B 
890 confirms that a report of completion of snapshot up- 
dating has been obtained from all of the storage devices 10 
B 1 90 that were designated and then proceeds to the 
next processing (step 1904, step 1905). 
[0082] Also, in the aforesaid processing, it was as- 
sumed that the storage device A 1 00 or storage device 
B 190 reported completion of processing in respect of 1$ 
the various types of instructions from the management 
software A 800 or management software B 890. How- 
ever, it would also be possible for completion of the var- 
ious types of processes by the storage device A 1 00 or 
storage device B 1 90 to be detected by the management 20 
software A 800 or management software B 890 by the 
management software A 800 or management software 
B 890 periodically making inquiries of the storage device 
A 1 00 or storage device B 1 90 regarding their process- 
ing condition in respect of the aforesaid instructions. 25 
[0083] Also, in the above processing, transfer 
processing of write data from the storage device A 1 00 
to the storage device B 190 is performed continuously, 
but it would be possible for the storage device A 1 00 to 
create a marker and to then stop transfer of write data 30 
and, in addition, for the storage device B 190, after de- 
tecting reflection processing of the received marker (af- 
ter reflection of the write data previous to the marker) to 
stop reflection of the write data i.e. to put the processing 
by the storage device A 1 00 and storage device B 1 90 35 
in a stopped condition (also called a suspended condi- 
tion). However, the storage device B 1 90 could perform 
write data reflection up to the detection of the marker 
without reference to instructions from the management 
software B 890. In this case, the marker creation instruc- 40 
tion isequivalentto an instruction to shift to the suspend- 
ed condition and mutually matched copies are created 
in the logical volume 500 of the storage device B 1 90 at 
the time where all of the storage devices B 190 have 
shifted to the suspended condition. When restarting the *5 
copy processing, the copy processing is recommenced 
by the storage device A 1 00 and storage device B 1 90 
in response to an instruction for recommencement of 
copy processing from the management software A 800 
or management software B 890 after acquisition of the so 
snapshot of the logical volume 500. As a result, copies 
having mutual consistency can be held in data stored 
by the snapshots, so MFB 690 or open system host B 
790 can continue business using the matched data. 
[0084] Also, in the processing described above, the 55 
various types of instructions, reports and exchange of 
information between the management software A 800 
or management software B 890 and storage device A 



1 00 and storage device B 1 90 may be executed by way 
of an I/O path 900 or could be executed by way of a 
network 920. In the case where instructions for marker 
creation are given in the form of a write request to the 
storage device A 100, a logical volume 500 that is not 
subject to the processing deferment of write instructions 
is provided at the storage device A 100 and the marker 
creation instructions are given in respect of this logical 
volume 500. 

[0085] In the above processing, the storage device A 
100 and storage device B 190 need not be connected 
in one-to-one relationship and it is not necessary that 
there should be the same number of devices, so long 
as the respective logical volumes 500 and logical vol- 
ume groups correspond as source and copy. 
[0086] Also, in the above construction, it was as- 
sumed that the management software A 800 was 
present in the MFA 600 and the management software 
B 890 was present in the MFB 690; however, it would 
be possible for the management software A 800 and 
management software B 890 to be present in any of the 
MFA 600, MFB 690, open system host A 700, open sys- 
tem host B 790, storage device A 1 00 or storage device 
B 1 90. Also, they could be present in another computer, 
not shown, connected with the storage device A 1 00 or 
storage device B 1 90. 

[0087] In the above processing, it was assumed that 
the write data reflection section B 240 determined the 
correct marker number, but it would also be possible for 
the correct marker number to be designated to the stor- 
age device B 190 as a parameter of the reflection in- 
structions by the management software B. Also, it could 
be arranged that when the management software A 800 
gives instructions for deferment of processing of write 
requests and marker creation to the storage device A 
100, a unique marker number is determined and desig- 
nated to the storage device A 100 and communicated 
to the management software A 890 and that this man- 
agement software B 890 then designates this marker 
number to the storage device B 1 90. 
[0088] In the above processing, the occasion at which 
the management software A 800 instructions for defer- 
ment of processing of write requests and marker crea- 
tion to the storage device A 100 may be determined in 
a manner linked with the processing of the APP 620 or 
APP 720. For example, synchronization of reflection to 
the copy may be performed at the checkpoint by giving 
instructions for deferment of write request processing 
and marker creation on the occasion of creation of a 
DBMS checkpoint. Business can therefore be continued 
by the MFB 690 or open system host B 790 using the 
data of this condition, by obtaining a snapshot in the con- 
dition in which the stored content of the source logical 
volume 500 at the checkpoint has been reflected to the 
copy in the target logical volume. 
[0089] It could also be arranged for the MFA 600 or 
open system host A 700 to defer issue of a write request 
to the storage device A 100 or to restart, by linking the 
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OS 610 or OS 710 with the management software A 
800, instead of the management software A 800 giving 
instructions for deferment of processing of write re- 
quests and canceling of deferment in respect of the stor- 
age device A 100. 

[0090] Also, as described in embodiment 1 , a logical 
volume for write data storage that is separate from the 
cache 400 could be prepared and the write data stored 
in this logical volume 500 for write data storage. Also, 
in the transfer processing of write data, it would be pos- 
sible for a write data transfer request to be initially issued 
in respect of the write data transfer section 220 by the 
write data reception section B 21 1 and for the write data 
to be transferred in respect of the write data reception 
section B 211 by the write data transfer section A 220 
after receiving this request. 

[0091] The processing described in this embodiment 
could also be implemented even if the write request 
does not contain a write time. 

[Embodiment 4] 

[0092] Figure 21 is a view showing an example of the 
layout of a computer system in embodiment 4. 
[0093] The difference with respect to Embodiment 3 
lies in that the MFA 600 and the open system host A 700 
are respectively connected with a plurality of storage de- 
vices C 1 80 by way of an I/O path 900 and the plurality 
of storage devices C 1 80 are connected with a plurality 
of storage devices A 1 00 by way of a transfer path 91 0. 
In addition, the plurality of storage devices C 180 are 
connected with another computer or device by means 
of a network 920. The storage device A 1 00 and the stor- 
age device B 1 90 of embodiment 4 have the same con- 
struction and function as the storage device A 100 and 
storage device B 190 in embodiment 3. 
[0094] In this embodiment, just as in the case of em- 
bodiment 2, a copy of the data stored in the logical vol- 
ume 500 of the storage device C 180 is stored in the 
logical volume 500 of the storage device A 1 00. Specif- 
ically, the storage device C 180 comprises the same 
construction and various types of information as in em- 
bodiment 2 and after receiving a write request to the log- 
ical volume 500 from the MFA 600 or open system host 
A 700, the storage device C 180 stores the write data 
that it has received and transfers this received write data 
to the write data reception section A 210 of the storage 
device A 1 00; however, it is then guaranteed that a copy 
of the write data 640 or write data 740 that was written 
by the storage device C 1 80 exists in the storage device 
A 100, by sending a write completion notification to the 
MFA 600 or open system host A 700 after waiting for a 
notification of completion of reception from the write da- 
ta reception section A 210, in the same way as in em- 
bodiment 2. 

[0095] In addition, a copy of the data stored in the log- 
ical volume 500 of the storage device C 180 is stored in 
a logical volume 500 of the storage device B 1 90 by the 



same processing as the processing described in em- 
bodiment 3. By processing as described above, as de- 
scribed in embodiment 2, even if for example some fault 
occurs in the storage device C 1 80 or in the transfer path 

5 910, causing transfer of data to the storage device A 
100 to become impossible, the expected content that 
was recognized as having been stored in the storage 
device C 180 when processing of the MFA 600 or open 
system host A 700 was interrupted can still be obtained 

10 from the storage device B 1 90, so the M FB 690 or open 
system host B 790 can continue business using this da- 
ta. 

[0096] In the above processing, the management 
software A 800 gives instructions for deferment of 

15 processing of write requests or marker creation or can- 
cellation of deferment of processing of write requests in 
respect of all of the storage devices C 180 in the same 
way as in the case of the processing performed in re- 
spect of the storage device A 1 00 in embodiment 3. Just 

20 as in the case of step 1600 of embodiment 3, the man- 
agement software A 800 first of all gives instructions for 
deferment of processing of write requests to all of the 
storage devices C 180. After receiving these instruc- 
tions, the write data reception section C 212 of the stor- 
es age device C 1 80 defers processing of write requests in 
the same way as in the case of the processing per- 
formed by the storage device A 100 in step 1601 and 
step 1 602 of embodiment 3 and reports commencement 
of deferment to the management software A 800. As de- 

30 scribed above, at this time, write data in respect of which 
a write completion notification has been given in respect 
of the MFA 600 or open system host A 700 has already 
been transferred to the storage device A 100 and the 
storage device A 100 creates write data management 

35 information 300 of this write data. In the same way as 
in the case of step 1603 and step 1604 of embodiment 
3, the management software A 800 confirms that a re- 
port of commencement of deferment has been obtained 
from all of the designated storage devices C 1 80 before 

40 proceeding to the following processing. 

[0097] Next, the management software A 800 gives 
instructions for marker creation to all of the storage de- 
vices C 1 80 in the same way as in the step 1 605 of em- 
bodiment 3. After receiving such an instruction, the stor- 

45 age device C 1 80 transmits a marker creation instruction 
through the path 91 0 or network 920 to the storage de- 
vice A 1 00 that stores the copy. After receiving the mark- 
er creation instruction, the storage device A 1 00 creates 
a marker in the same way as in step 1606, step 1607 

50 and step 1 608 of embodiment 3 and reports completion 
of marker creation to the storage device C 180 through 
the transfer path 91 0 or network 920. After receiving the 
report, the storage device C 180 reports completion of 
marker creation to the management software A 800. 

55 The management software A 800 confirms that a report 
of completion of marker creation has been received from 
all of the designated storage devices C 1 80 in the same 
way as in step 1609 and step 1610 of embodiment 3 
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before proceeding to the next processing. 
[0098] Next, the management software A 800, in the 
same way as in step 1611 of embodiment 3, gives in- 
structions for cancellation of deferment of processing of 
write requests to all of the storage devices C 180. After 
receiving these instructions, the write data reception 
section C 212 of the storage device C 180 cancels the 
write request processing deferment in the same way as 
the processing that was performed by the storage de- 
vice A 100 in step 1612 and step 1613 of embodiment 
3 and reports this cancellation of deferment to the man- 
agement software A 800. 

[0099] Specifically, deferment of processing of write 
requests and cancellation of deferment are performed 
by the storage device C 1 80 and marker creation mean- 
while is performed by the storage device A 1 00 on trans- 
mission to the storage device A 100 of an instruction by 
the storage device C 1 80. As described above, write da- 
ta in respect of which completion of writing has been 
notified to the MFA 600 or open system host A 700 has 
already been transferred to the storage device A 100 
and write data management information 300 of such 
write data is created in the storage device A 1 00, so de- 
ferment of processing of write requests by the storage 
device A 100 in embodiment 3 and deferment of 
processing of write requests by the storage device C 
180 in this embodiment are equivalent. Consequently, 
by performing processing as described above and by 
performing other processing as described in embodi- 
ment 3, in the construction of this embodiment, reflec- 
tion of updating to the copies can be synchronized at 
the marker time by stopping data updating by the MFA 
600 and open system host A 700 in the same way as in 
embodiment 3 and creating a marker of the updated 
condition with unified timing (checkpoint) between the 
plurality of storage devices; mutual consistency of the 
respective copies with the mainframe host data and the 
open system host data can thus be achieved at this time. 
Furthermore, mutually matched copies are maintained 
in snapshot volumes by acquiring snapshots at the time 
of synchronization of reflection and the M FB 690 or open 
system host B 790 can therefore continue business us- 
ing matched data. 

[0100] In the above processing, it was assumed that 
the management software A 800 gave instructions for 
marker creation to the storage devices C 180 and the 
storage devices C 180 transmitted these instructions to 
the storage devices A 100; however, it would also be 
possible for the management software A 800 to give in- 
structions for marker creation directly to all of the stor- 
age devices A 1 00 and for the storage devices A 1 00 to 
report completion of marker creation to the manage- 
ment software 800. Specifically, the management soft- 
ware A 800 first of all gives instructions for deferment of 
write request processing to all of the storage devices C 
1 80 and the management software A 800 confirms that 
reports of commencement of deferment have been re- 
ceived from all of the designated storage devices C 1 80 
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before giving instructions for marker creation to ail of the 
storage devices A 1 80 in the same way as in step 1 605 
of embodiment 3. After having received these instruc- 
tions, the storage device A 1 00 creates a marker in the 
s same way as in step 1 606, step 1 607 and step 1 608 of 
embodiment 3 and reports completion of marker crea- 
tion to the management software 800. After confirming 
that reports of completion of marker creation have been 
obtained from all of the designated storage devices A 
10 1 00 in the same way as in step 1 609 and step 1 61 0 of 
embodiment 3, the management software A 800 may 
be arranged to give instructions for the cancellation of 
deferment of write request processing to all of the stor- 
age devices C 1 80. 
15 [0101] Also, it would be possible that the storage de- 
vices C 1 80 are provided with a marker creation section 
and marker number information 330 and create a mark- 
er on receipt of instructions for marker creation from the 
management software A 800; the marker, which has 
been created as write data, is then transferred to the 
storage device A 1 00 and completion of marker creation 
may be arranged to be reported to the management 
software A 800 when a report of receipt thereof has been 
received from the write data reception section 21 0 of the 
storage device A 100. In this case, the storage device 
A 1 00 treats the received marker as a special type of 
write data, which is transferred to the storage device B 
190 after processing in the same way as ordinary write 
data except that reflection to the copy is not performed. 
[0102] In any case, the above can be implemented ir- 
respective of the number of storage devices C 1 80 that 
are connected with the storage devices A 100 and de- 
posit copies on the storage devices A 100. 
[0103] Also, although not shown, if a mainframe host 
and open system host are connected with the storage 
devices A 1 00 by an I/O path, if for example some fault 
occurs in the MFA 600 or open system host A 700 or 
storage devices C 180, the aforesaid mainframe host 
and open system host can continue business using the 
content of the logical volume 500 of the storage device 
A 1 00 that is matched therewith. 
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Claims 

1. A system for copying data between a plurality of 
storage systems, comprising: 

a first storage system (100, 180) coupled to a 
plurality of computers (600, 700), which com- 
prises a first logical volume (500) storing data 
received from the plurality of computers; and 
a second storage system (1 00, 1 90) coupled to 
said first storage system, which comprises a 
second logical volume (500) storing copy data 
of data stored in said first logical volume; 

wherein said first storage system assigns time 
information to write data received from the plurality 
of computers and sends the write data and time in- 
formation to said second storage system; and 

said second storage system stores write data 
received from said first storage system in said sec- 
ond logical volume in an order based on the time 
information assigned to this write data. 

2. The system according to claim 1 , wherein 

said first storage system (100, 180), when a 
write time is assigned to the write request or write 
data received from the computer (600, 700), 
records the received write time and sends the write 
data with the received write time to said second 
storage system (100, 190), and 
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when a write time has not been applied to the 
write request or write data received from the com- 
puter, assigns the write time recorded by said first 
storage system to the received write data and sends 
the write data with the assigned write time to said 
second storage system. 

3. The system according to claim 2, wherein 

said first storage system (100, 180) compris- 
es a plurality of first logical volumes (500), 

said second storage system (1 00, 190) com- 
prises a plurality of second logical volumes (500), 

each of said plurality of second logical vol- 
umes belongs to one of a plurality of logical volume 
groups, and 

said second storage system, in respect of 
each of said plurality of logical volume groups, 
records the latest write time in the write times as- 
signed to the write data stored in a logical volume 
in the logical volume groups and stores write data, 
to which a write time indicating a time prior to the 
latest write time has been assigned, in said second 
logical volume. 

4. The system according to claim 3, wherein 

said first storage system (1 00, 1 80) further as- 
signs a sequential number to the write data for each 
of the logical volume groups and sends the write 
data with the assigned sequential number to said 
second storage system (100, 190), and 

said second storage system stores write data 
in a second logical volume in the order of the se- 
quential numbers assigned to the write data for 
each of the logical volume groups. 

5. The system according to claim 4, wherein 

said second storage system (1 00, 1 90) stores 
write data in the sequential number order, so that 
there is no missing in the sequential numbers that 
are assigned to the write data stored in the second 
logical volume, for each logical volume group. 

6. The system according to claim 1 , wherein 

the time information that is assigned to the 
write data by said first storage system (1 00, 1 80) is 
a sequential number applied to the write data in the 
order of reception of the write data. 

7. A system for copying data between a plurality of 
storage systems, comprising: 

a first storage system (1 80) coupled to a plural- 
ity of computers (600, 700), which comprises a 
first logical volume (500) storing data received 
from said plurality of computers; 
a second storage system (100) comprising a 
second logical volume (500), which stores copy 
data of data stored in said first logical volume; 



and 

a third storage system (1 90) comprising a third 
logical volume (500), which stores copy data of 
data stored in said first logical volume; 

5 

wherein said first storage system stores write 
data received from said plurality of computers in 
said first logical volume and sends the write data 
received from plurality of computers to said second 

io storage system, 

said second storage system assigns time in- 
formation to the write data received from said first 
storage system and sends the write data with the 
time information to said third storage system; and 

*5 said third storage system stores write data re- 

ceived from said second storage system in said 
third logical volume in accordance with the time in- 
formation assigned to the write data. 

20 8. The system according to claim 7, wherein 

the time information assigned to the write data 
is a sequential number assigned to the write data in 
the order of reception of the write data. 

25 g. The system according to claim 7. wherein 

said first storage system (180) sends a com- 
pletion report to the computer (600, 700) after re- 
ceiving a report of reception of write data from said 
second storage system (100). 

30 

10. The system according to claim 7, wherein 

said first storage system (180), when a write 
time is assigned to the write request or write data 
received from the computer (600, 700), records the 
35 received write time and sends the write data with 
the write time to said second storage system (1 00), 
and 

when a write time is assigned to the write data 
received from said first storage system, said second 
40 storage system records the received write time and 
sends the write data with the write time to the third 
storage system (190), 

when a write time has not been assigned to 
the write data received from said first storage sys- 
45 tern, said second storage system assigns the write 
time recorded in said second storage system to the 
received write data and sends the write data with 
the write time to said third storage system, and 
said third storage system stores the received 
50 write data in said third logical volume in accordance 
with the write time assigned to the write data. 

11. The system according to claim 10, wherein 

said second storage system (100) comprises 
55 a plurality of second logical volumes (500), 

said third storage system (190) comprises a 
plurality of third logical volumes (500), 

said plurality of second logical volumes and 
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said plurality of third of logical volumes respectively 
belong to one of a plurality of logical volume groups, 

said third storage system, in respect of each 
of said plurality of logical volume groups, records 
the write time that is closest to the current time in 5 
the write times assigned to the write data stored in 
a logical volume in the logical volume group, and 

stores write data, to which a write time indi- 
cating time prior to the recorded write time has been 
assigned, in said third logical volume. io 

12. The system according to claim 11, wherein 

said second storage system (100) further as- 
signs a sequential number to the write data in re- 
spect of a logical volume (500) belonging to a logical '5 
volume group for each of the logical volume groups 
and sends the write data with the sequential number 
to said third storage system, and 

said third storage system (190) stores write 
data in a third logical volume in the order of the se- 20 
quential numbers assigned to the write data. 

13. A system for copying data between a plurality of 
storage systems, comprising: 

25 

a first storage system (100, 180) coupled to a 
plurality of computers (600, 700), which com- 
prises a first logical volume (500) storing write 
data received from the plurality of computers; 
and 30 
a second storage system (1 00, 1 90) coupled to 
said first storage system, which comprises a 
second logical volume (500) storing copy data 
of data stored in said first logical volume; 

35 

wherein said first storage system assigns a 
sequential number to write data received from the 
computer, sends the write data with the sequential 
number to said second storage system, creates a 
marker including a sequential number, and sends 40 
the marker to said second storage system, and 

said second storage system stores write data, 
to which a sequential number smaller than the se- 
quential number included in the received maker is 
assigned, in said second logical volume in the order 45 
of the sequential numbers applied to the write data. 

14. The system according to claim 13, wherein 

said first storage system (100, 180) defers 
processing of write requests received from the com- 50 
puter (600, 700) based on an instruction from the 
computer, creates a marker to which a sequential 
number is included based on an instruction from the 
computer, sends the marker to said second storage 
system (100, 190), and recommences processing 55 
of deferred write requests after sending the marker. 

15. The system according to claim 13, wherein 



said first storage system (100, 1 80), while is- 
suance of a write request to said first storage sys- 
tem from the computer (600, 700) is deferred, cre- 
ates a marker including a sequential number and 
sends the marker to said second storage system 
(100, 190). 

16. The system according to claim 13, further compris- 
ing: 

a plurality of first storage systems (100, 180); 
and 

a plurality of second storage systems (100, 
190); 

wherein each of the plurality of first storage 
systems creates a marker having a sequential 
number in accordance with an instructions from the 
computer (600, 700) and sends the created marker 
to one of said plurality of second storage systems, 
and 

each of the plurality of second storage sys- 
tems stores write data, to which a sequential 
number smaller than the sequential number includ- 
ed in the received marker is assigned, in the second 
logical volumes (500) in the sequential number or- 
der. 

17. The system according to claim 16, 

wherein said plurality of first storage systems 
(100, 180) comprise a plurality of first logical vol- 
umes (500), 

said plurality of second storage systems (1 00, 
1 90) comprise a plurality of second logical volumes 
(500), 

said plurality of first logical volumes and said 
plurality of second logical volumes respectively be- 
long to one of a plurality of logical volume groups, 

said first storage system assigns a sequential 
number to write data for each logical volume group, 
sends the write data with the sequential number to 
said second storage system, creates a marker hav- 
ing a sequential number for each logical volume 
group, and sends the marker to said second storage 
system, and 

said second storage system stores write data, 
to which sequential number smaller than the se- 
quential number included in the received maker for 
the corresponding logical volume group is as- 
signed, in a second logical volume. 

18. A system for copying data between a plurality of 
storage systems, comprising: 

a first storage system (1 80) coupled to a plural- 
ity of computers (600, 700), which comprises a 
first logical volume (500) storing data received 
from the plurality of computers; 
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a second storage system (1 00) coupled to said 
first storage system, which comprises a second 
logical volume (500) storing copy data of data 
stored in said first logical volume; and 
a third storage system (190) coupled to said 5 
second storage system, which comprises a 
third logical volume (500) storing copy data of 
data stored in said first logical volume 

wherein 10 
said first storage system stores write data re- 
ceived from the computer in said first logical volume 
and sends the write data to said second storage 
system, 

said second storage system assigns a se- '5 
quential numberto the write data received from said 
first storage system, sends the write data with the 
sequential numberto said third storage system, cre- 
ates a marker including a sequential number, and 
sends this marker to said third storage system, and 20 

said third storage system stores the write da- 
ta, to which a sequential number smaller than the 
sequential number included in the received maker 
is assigned, in said third logical volume in the order 
of the sequential numbers assigned to the data. 25 

1 9. The system according to claim 1 8, wherein said first 
storage system (1 80) defers processing of write re- 
quests received from the computer (600, 700) 
based on an instruction received from thecomputer, 30 

said first storage system transfers to said sec- 
ond storage system an instruction for creation of a 
marker received from the computer, and 

said second storage system creates a marker 
based on the instruction for creation of a marker re- 35 
ceived from said first storage system. 

20. The system according to claim 18, wherein 

said second storage system (100) creates a 
marker while issuance of write requests is deferred *o 
by a computer (600, 700). 
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